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ABSTRACT 

An attitude questionnaire, administered to third, 
fourth, and fifth grade students, is discussed. The questionnaire, 
"How I Feel," was composed of a packet of directions to the teacher, 
directions to the student with sample questions, and 11 concepts, 
each of which was evaluated by six bipolar adject-ive pairs that were 
chosen because of their relevance to the type of information desired 
by the researchers and by their high loading on Osgood's evaluative 
dimension. Nine of the concepts were scored whereas two served as 
validity-type scales to identify those students who did not 
understand what they were asked to do or who may have filled in the 
blanks with random checks. The students to whom the questionnaire was 
administered were 238 who were attending a school using Individually 
Prescribed Instruction (IPI) . Each of the individual bipolar 
adjective pair scales was scored along a five-point continuum. The 
questionnaire was administered again to 201 of the original subjects 
seven months following the first administration, using the same 
instructions but a different "special" teacher. Results showed the 
coefficient alpha reliabilities for the individual scales ranged from 
.86 to .92. A factor analysis of the entire instrument indicated that 
the six bipolar adjective pairs used to evaluate the nine concepts 
were essentially measuring the same factor and could therefore be 
summed, thus reducing the original 54— items to 9 items. The 
test-retest reliabilities for six of the nine questions ranged from 
.35 to .44; they were considerably lower for the other three 
questions. Tables present the data, and appendixes provide the 
questionnaire and related materials. (DB) 



tD mw 



U S. DEPARTMENT OF HEALTH. 
EDUCATION & WELFARE 
OFFICE OF EDUCATION 
THIS DOCUMENT HAS BEEN REPRO- 
DUCED EXACTLY AS RECEIVED FROM 
THE PERSON OR ORGANIZATION ORIG- 
INATING IT. POINTS OF VIEW OR OPIN- 
IONS STATED DO NOT NECESSARILY 
REPRESENT OFFICIAL OFFICE OF EDU- 
CATION POSITION OR POLICY 



THE RELIABILITY AND USABILITY OR A SEMANTIC DIFFERENTIAL ATTITUDE 
SCALE WITH THIRD THROUGH FIFTH GRADE STUDENTS * 



Dr. Henty J. 01.es 
Southwest Texas State University 

Dr 0 John G. Botvin 

Learning Research ain* Development Center 
University o.t Pittsburgh 



75 




A Paper Presented To the 1972 Convention of the 
Nflcionr! Council on Myas :reraeni: in Education 
Chicago, 111 iiioia 




*] ; -.'ised upon 
Lniver-ity 



r. por : . ion of a PhD uisserCatiou 
of Pi.(.i. jlr.irgli, Pittsburgh, P<> 0 



c c*iuole :ed n 'c 



the 



FILMED FROM BEST AVAILABLE COPY 



The semantic differential has become a very popular and useful measure- 
ment technique in psychological and educational research for the study of 
attitudes,, However, it has seldom been used with children of elementary 
school age, primarily because it was intuitively assumed that they would 
not be able to understand how to answer the questions which would result in 
unreliable and therefore useless results. 

The attitude questionnaire used in this study, "How I Feel" was a specially 
adapted version of a semantic differential type instrument first developed and 
used by Research For Better Schools, Inc,, of Philadelphia, Pa 0 Although this • 
questionnaire was specifically tailored to gather information about a par- 
ticular instructional program, its form could easily be generalized. The 
"How I Feel" questionnaire (Appendix A) was composed of a packet of directions 
to the teacher, directions to the student with sample questions, and eleven 
concepts, each of which was evaluated by six bipolar adjective pairs that were 
chosen because of their relevance to the type of information the researchers 
wished to gather and their high loadings on Osgood's (Osgood, 1967) evaluative 
dimension. Nine of the concepts were actually scored while two served as 
validity type scales to identify those students who did not understand what 
they were asked to do or who may have filled in the blanks with random checks. 
The two validity questions were: 1. Being Punished For No Reason Is; 2. Being 
Rewarded For Doing Something Well Is, Students who answered the first question 
with predominantly positive answers or the second question with predominantly 
negative answers would have been eliminated. None of the students were dropped 
for this reason. Several students skipped items. but they were able to complete 
the questionnaire the following day. 



2 



L 



r. 

& 



V : | 

s'-.' 



P 



n: 



w\ 






The questionnaire was administered by a 'special' teacher to 238 third, 
fourth, and fifth grade students in an elementary school using Individually 
Prescribed Instruction (IPIJ^a program developed by the Learning Research and 
Development Center of the University of Pittsburgh, The teacher carefully 
read the directions to the students, answered their questions, and observed 
their completion of the questionnaire. To help prevent students from answering 
the questions in a way they felt their teacher would expect them to respond, 
they were clearly told both orally and in the printed instructions that their 
teacher or principal would not see their papers. The regular classroom teacher 
was asked to leave the room until her students completed the scale, 

Rach of the individual bi-polar adjective pair scales was scored along a 
five point continuum. One point was given to the most negative response and 
five points co the most positive. In an attempt to prevent response patterning, 
the direction of the scales as well as the order of the bipolar adjective pairs 
were randomly assigned. 

Table 1 presents the coefficient alpha, concept mean, sum of the item 
variances, and total variance for each of the nine concepts scored. The values 
of coefficient alpha range from a low of ,74 to a high of .92, Table 2 lists 
reliability data for the instrument first looking at it as one 54-item test 
and then after collapsing over scales within concepts, viewing it as one 9-item 
test. The subjects in this case are separated by grade. As expected, coefficient 
alpha is considerably higher for the 54-item test than for the summation 9-itera 
test. With the exception of the third grade 9-item coefficient alpha, the 
values are very similar for each of the three grades. 

Table 3 lists the total mean (mean of all the ratings for all concepts) sum 
of the item variances, and the total variance for each the 54-item test and the 
9-item teat for all three grades. There is relatively little difference among 
the values across grades except that third grade students exibited slightly 
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more positive attitudes, a lower sum of item variances, and a higher total 
variance than fourth or fifth grade students. 

Appendices B, C, and D present the means and standard deviations, correlation 
matrices, factor Eigenvalues, and chi square te3ts of sphericity from principal 
components analysis, and rotated factor patterns for the six scales that evaluate 
each of the nine concepts that compose the student attitude questionnaire. A 
review of the means in Appendix 15 shows that for the first seven questions, all 
of the means are above, and in many cases, considerably above the mean or neutral 
point for the scale. This can be interpreted to indicate that IPX students 
generally display very positive attitudes, as 'measured by this instrument, toward 
school, their math teacher, IPI related activities in math, and themselves. 

They feel that these activities are generally good, useftjl , honest, interesting, 
fair, and happy. The means for the last two questions are considerably lower ^ 
which may show that the students feel that failing a math CET (test) is bad, 
useless, and sad. However, they are not sure whether or not it is honest or 
dishonest or fair or unfair. All of the means for Question 9, Copying Workpage 
Answers From the Key^are below the neutral point. 

A general rule of thumb in factor analysis is to consider primarily only 
those factors with Eigenvalues greater than 1 for interpretation thmugh factor 
rotation. Appendix C shows that the scales for 8 out of 9 of the concepts 
had only one Eigenvalue greater than 1. In each case, the first principal 
components factor accounted for from 44.4 to 71.9°/. of the variance f which 
indicates that the scales share much of the variance or that they have a common 
factor. Again j this would be expected since all of the scales were selected because 
they shared high loadings on the evaluative dimension in previous research. 

In an attempt to gain a better understanding of the factorial composition 
of the scales, other criteria including the factor's percent trace contribution, 
and its chi square for significance were considered in addition to the factor's 
Eigenvalue when deciding whether or not to preserve a factor for rotation. In 
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each case, enough factors to account for 70 to 80% of the variance were 
rotated. Thi^ procedure is supported by Lohne;.; (1966)jv:ho warns that there 
is probably greater danger in underfactoring than in overfactoring. 
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Appendix D presents the rotated factor patterns for the number of 
factors selected from principal components analysis for each concept except number 
aeven^in which case the first principal components factor accounted for 71 . 9 % 
of the variance ,which indicated that rotation with additional factors would not 
be helpful. As expected, after reviewing the relatively high intercorrelations 
among the scales and the amount of variance accounted for by the first principal 
components factor, it was difficult to develop a clear interpretation of the 
rotated factor patterns. The following list of: names were assigned to scales 
that had consistently high loadings on selected factors in an attempt to attach 
some interpretive meaning to the rotated factors. 

1. General Evaluative Factor 

2. Feeling or Emotional Tone 



3. Moral Judgement 
4o Personal Interest 
5 0 Utility 



For various concepts, all six scales 
had high loadings 

Usually, high loadings were found on the 
good-bad, happy-sad, and interesting-boring scale 
Hone s t - d i sli on e $ t and fair-unfair 
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Interesting-boring and happy- sad 
Useful-useless 

A principal components analysis with variroax rotation was also conducted 
on the questionnaire viewing it as one 54- item test and completely disregarding 
the nine subsections. Appendix E shows the Eigenvalues, percent trace and 
chi-square values lirom principal components analysis for the thirteen factors 
that had Eigenvalues greater than one. The first factor accounts for 21,0% 
of the variance while the 13th accounts for only 1,9%. These thirteen factors 
were subject to varimax rotation. Hie resulting factor pattern shown in Appendix 
F clearly shows that the test divides it self into nine subtests. The six bi- 
polar adjective pairs for each subtest c/.; concept generally loaded on the same 
factor while the nine subtests with their associated adjective pairs loaded on 
different factors. For example, looking at the rotated factor pattern in 



Appendix F, tae six bipolar adjective pairs for question one all had high loadings 
only on factor VIII. The six bipolar adjective pairs for question two all had 
their highest loadings on factor VII. This pattern apparently indicates that 
the six bipolar adjective pairs are measuring essentially the same factor while 
the individual concepts are measuring different factors. 

Because the six bipolar adjective pairs are relatively unifactorial for 
each concept and the concepts themselves each load on a different factor as 
shown in the pretending analyses, the six ratings for each concept were summed. 
These summations were then analyzed ns though they were obtained from one, 9-item 
test. 

Tables 4 , 5, and n present the means, standard deviations, and correlation 
matrix. Eigenvalues, percent trace, and chi-square values from principal components 
analysis for the first five factors, and the varimax rotation of those factors. 

The intercorrelations in Table 4 are all relatively low 0 Hie highest correlation 
(0,59) is betv/een concepts or variables 4, Taking A Math CET Is, and 5, IPI 
Math Class Is. 

Table 5 lists the result of a principal components analysis on the 

j 

correlation matrix for all nine variables. The first three factors extracted 

each had an Eigenvalue greater than one.and together they accounted for 61 .4% 

\ * 

| of the variance. Two additional factors, A and 5, accounted for another 17% 

of the variance. 

• Varimax rotations were performed seeking three to seven rotated factors. 

t 

The results of a five factor rotation (Table 6) produced communal ities ranging 
\ from 0.67 to 0.95 indicating that five factors accounted for most of the variance 

| In each test. The five factor rotation also produced the mast interpretable factor 

f 

{ 

s pattern with the least number of factors. 

1 

y FACTOR 1: General School Evaluative Factor (28% of the variance) 

r- 

i Five concepts had medium to high loadings on this factor. They included: 



Working on Math Problems Is; My School Is; Taking A Math CET Is; My 
Math TeacheJr Is. 

FACTOR II: Propensity To Copy From Key (13% of the variance) 

One concept. Copying Work Page Answers From The Key Is, had a high 
positive loading (.396) on this factor indicating that students who had 
high scores on this factor had positive attitudes (good, useful, honest, 
etcetra) toward copying. Other loadings, although not particularly high, 
indicated that a student who had a high score on the copying factor also was 
likely to have a positive attitude toward taking a math CET. He was likely 
to have negative attitudes toward working on math problems, his school, and 
his math teacher, 

FACTOR III; Independent Activities (12% of the variance) 

One concept, Scoring My Own Math Work Page had a high negative loading 
(-.907) on this factor indicating that students with high scores on this 
factor had negative attitudes toward scoring their o\m work pages. They 
also showed negative attitudes toward themselves and taking math CET f s, 

FACTOR IV; Unnamed (15% of the variance) 

Students with high scores on this factor had negative attitudes toward 
their math teachers, themselves, th cir school and IPI math class, 

FACTOR V: Failure (12% of the variance) 

Students with a high score on- this factor expressed negative attitudes 
toward failing a math CET and working on math problems. 

A five element factor score vector was computed from the rotated factor 



pattern matrix for each of the subjects for use in subsequent analysis, 
Cooley and Lohnes (1971) suggest that the use of factor scores in place of 
raw scores can lead to more understandable results* 



The questionnaire was again administered to 201 of the original 234 
subjects in December, 1971, seven months after the first administration 
using the same instructions but a different 'special' teacher. Table 7 
presents the test-retest reliability coefficients for the summation scores 
for each question. Six of the coefficients range from <,35 to ,44jWhich are 
surprisingly High considering the instability of elementary students' attitudes 
and the long time lapse. The lower reliability coefficients for the remaining 
three questions were intuitively predictable. The reliability coefficient for 
My School Is was .18. There had been a number of changes in the school 
(new teachers, new rules, etc.) since the test was first given. Overall, the 
students had a slightly more negative opinion of their school in December than 
they had in May. The correlation for My Math Teacher Is was essentially zero 
(.08). All of the students had new math teachers. The correlation for Copying 
Work Page Answers From The Key Is was also zero (.04). As a result of a 
research study completed at this school after the first administration of the 
attitude questionnaire which showed that the vast majority of the students 
were indeed misusing self-scoring or copying work page answers from the key, the 
teachers and principal made a continuing effort since September, 1971, to inform 
the students of the necessity of proper self-scoring and to detect improper 
self-scoring when it occurred. The students' attitudes toward improper self- 
scoring were slightly more negative on the second administration. 

CONCLUSIONS 

The semantic differential questionnaire used in this study with third, 
fourth, and fifth grade students was able to collect a relatively large amount 
of data in a reasonable period of time that could be analyzed by a variety 
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of readily available "package" computer programs. The coefficient alpha 

t 

reliabilities for the individual scales ranged from .86 to ,92. A factor 
analysis of the entire instrument indicated that the six bi-polar adjective 
pairs used to evaluate each of the nine concepts were essentially measuring 
the same factor and therefore could be summed^ thus reducing the original 
54-item questionnaire to a 9-item questionnaire. The entire questionnaire was 
again administered to the same students seven months later during a new 
school year. The test-retest reliabilities for six of the nine questions ranged 
from .35 to .44. The reliability coefficients for the other three questions 
were considerably lower. However, these lower reliability coefficients could 
be explained by actual changes that had occurred during the waiting period. 

This study has shown that the semantic differential, properly presented, 
can be a reliable and usable measure of attitudes in students as young as 
third grade. 






TABLE 1 



COEFFICIENT ALPHA, TEST MEAN, SUMS OF THE 
ITEM VARIANCES, AND TOTAL VARIANCE FOR 
EACH OF THE NINE SUBTESTS THAT COMPRISE 
THE SEMANTIC DIFFERENTIAL 



Concept 


Coefficient 


Mean 


Total 


Sum of Item 




A Ipha 




Variance 


Variances 


1 Working On Math Problems Is 


. . .85 


21.24 


34.72 


10.19 


2 My School Is 




24.92 


27.62 


8.07 


Scoring My Own 

3 Math Work Pages Is 




25.52 


19.40 


6.25 


4 Taking A Math CET Is 




25.48 


21.28 


6.48 


5 IPI Math Class Is 




24. 12 


34.08 


9.15 


6 I Am 




25,21 


16.43 


6.26 


7 My Math Teacher Is ...... . 




25. 14 


37 . 06 


8.67 


8 Failing A Math CET is 




13.07 


27.11 


9.34 


Copying Work. Page Answers 
9 From The Key Is 




10.72 


41.50 


10.10 



TABLE 2 



COEFFICIENT ALPHA FOR THE SEMANTIC DIFFERENTIAL 
VIEWED AS BOTH A 54-ITEM TEST AND AFTER SUMMING 
WITHIN CONCEPTS AS A 9-ITEM TEST FOR GRADES 
THREE , FOUR , AND FIVE 



Coefficient Alpha 



Grade 


Raw Scores (54 Items) 


Summed Scores (9 Items ) 


Third 


.86 




.54 


Fourth 


.92 




.72 


Fifth 


.91 




.71 



TABLE 3 



MEANS . SUM OF ITEM VARIANCES AND TOTAL VARIANCE 
FOR THE SEMANTIC DIFFERENTIAL , GRADES THREE 

FOUR , AND FIVE 



Grade 


Number Of 

Items 


Mean 


Total 

Variance 


Sum of Item 

Variances 


Third 


54 


198 . 18 


502. 69 


81.71 


Third 


9 


198.18 


502.69 


262.96 


Fourth 


54 


195. 59 


708. 76 


71.14 


Fourth 


9 


195. 59 


708. 76 


257.37 


Fifth 


54 


193.36 


636.27 


70.49 


Fifth 


9 


193.36 


636.27 


241.85 



TABLE 5 



PRINCIPAL COMPONENTS ANALYSIS OF THE 
SUM ELATION SCORES FOR EACH OF THE 
NINE CONCEPTS OF THE SEMANTIC 

DIFFERENTIAL 



Factonr 


Eigenvalue 


Percent Trace 


Cum Percent 


N.D.F. 


Chi-Square 


1 


3. 1224 


34.7 


34.7 


36 


502.43 


2 


1,2756 


14.2 


48.9 


28 


192. 29 


3 


1.1260 


12.5 


61.4 


21 


139.30 


4 


0.8598 


9.6 


70.9 


15 


87.89 


5 


0.8045 


8.9 


79.9 


10 


61.27 



ROTATED FACTOR PATTERN 
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TABLE 7 



TEST RETEST RELIABILITY COEFFICIENTS FOR THE NINE CONCEPTS 
THAT COMPRISE THE SEMANTIC DIFFERENTIAL' QUESTIONNAIRE 



=====s=====r3=======:===:==r3====z3=ra==== 
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Concept 



Test-Retest Reliability 



1. Working On Math Problems Is .36 

2 0 My School Is .18 

3. Scoring My Own Math Work Pages Is .40 

4. Taking A Math Test Is .35 

5. IPI Math Class Is .43 

6. I am # 44 

7. My Math Teacher Is .08 

8. Failing A Math Test Is .38 

9. Copying Workpage Answers From The Key Is .04 
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John feel* that WATCH 1 Wi TV IS vary HAMT. He can 
*h©w that he feels ViATTHIMC TV IS very HAPPY by nakln^ 
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APPENDIX E 

PRINCIPAL COMPONENTS ANALYSIS FOR SEMANTIC 
DIFFERENTIAL RAW SCORES 
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